Parallelization of the HIROMB Ocean Model

The HIROMB model of the North Sea and the Baltic Sea has been running operationally at SMHI since 1995 and delivers daily forecasts of currents, temperature, salinity, water level, and ice conditions. We have parallelized and ported HIROMB from vector computers to distributed memory parallel computers like CRAY T3E and SGI Origin 3800. The parallelization has allowed grid resolution to be refined from 3 to 1 nautical miles over the Baltic Sea, without increasing the total runtime. The grid decomposition strategy is designed by Jarmo Rantakokko. It is blockbased and computed with a combination of structured and unstructured methods. Load balance is fine-tuned by assigning multiple block to each processor. We analyze decomposition quality for two variants of the algorithm and suggest some improvements. Domain decomposition and block management routines are encapsulated in F90 modules. With the use of F90 pointers, one block at a time is presented to the original F77 code. We dedicate one processor for performing GRIB formatted file I/O which is fully overlapped with model time stepping. Computation time in winter season is dominated by the model’s viscous-viscoplastic ice dynamics component. Its parallelization was complicated by the need for a direct Gaussian elimination solver. Two parallel direct multi-frontal sparse matrix solvers have been integrated with parallel HIROMB: MUMPS and a solver written by Bruce Herndon. We have extended and adapted Herndon’s solver for HIROMB, and use ParMETIS for parallel load balancing of the matrix in each time step. We compare the performance of the two solvers on T3E and Origin platforms. Herndon’s solver runs all solver stages in parallel which makes it the better choice for our application. ISBN 91-7283-296-7 • TRITA-NA-0210 • ISSN 0348-2952 • ISRN KTH/NA/R--02/10--SE